Skip to content

docs: send access tokens in the TES tutorials - #135

Merged
uniqueg merged 6 commits into
mainfrom
docs/134-tes-tokens
Nov 13, 2026
Merged

uniqueg merged 6 commits into
mainfrom
docs/134-tes-tokens

Conversation

@uniqueg

@uniqueg uniqueg commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

Summary

The TES tutorials (docs/tutorials/tes-services/) now work with TES services that require access tokens.

  • New tes-hub/tes-api.sh, sourced by tes-service.sh and omop-tesk-fi.sh: every request to the TES goes through tes_curl. When TES_TOKEN is set, tes_curl adds Authorization: Bearer $TES_TOKEN. The header reaches curl as -H @<(printf ...): the builtin printf writes it into a pipe, so the token is never on a command line, in a file or in a log. Without TES_TOKEN, no header is sent.

  • Fail early: every command that uses the TES first lists one task. On 401 (no token, or an invalid or expired one) or 403 (valid token, but the account is not in the compute group), it stops with a clear message. down is the exception: it stops the keep-alive loop and locks the old tasks out of the hub without the TES, then warns that it couldn't cancel them. restart checks the token before it takes anything down. tes-hub.sh unsets TES_TOKEN, also when started by hand.

  • Keep-alive loops: a loop logs a rejected token to .state/watch.log and sends no new tasks. Running up with TES_TOKEN set replaces a running loop, so the loop picks up a new token. Tokens live about 4 hours. The loop of tes-service.sh only stops between rounds, never inside lock_out.

  • Token kept away from other processes: the launchers un-export TES_TOKEN first thing. A token goes over HTTPS only (plain HTTP only to localhost), and curl runs with -q, so that ~/.curlrc is ignored. The launchers pass it only to their own loop, so the hub (chisel, which runs for days) doesn't inherit it. TES_TOKEN is also excluded from the task environment built from service.conf.

  • Docs: a new "Get an access token" section covers:

    • LS Login and the ELIXIR On-Cloud compute group ECP_CLN:EOC-Compute (ECP_CLN:ADMIN may run tasks too);
    • the token lifetime of about 4 hours;
    • a hidden read -rs prompt for the token;
    • a table of what 401 and 403 mean;
    • the behaviour of the loops.

    Further doc changes:

    • The commands page sends the token through stdin in its curl commands.
    • tes-api.sh is added to the install block and the file table.
    • TES_TOKEN is added to the settings.
    • The troubleshooting list has an entry for 401 and 403.
    • The prerequisites now say curl 7.55 or newer.
    • The danger box no longer says that the TES has no authentication; it is now worded generically.
    • A note says that the default TES_URL will change.

Closes #134

How it was verified

  • bash -n on all scripts: OK.
  • shellcheck -x -e SC1091 (0.11.0) on tes-service.sh, tes-api.sh and omop-tesk-fi.sh: clean. SC1091 is info-level and concerns runtime files such as .state/config and service.conf, as before.
  • Mock TES that requires tokens (Python), with a stand-in for the hub (no chisel or nginx). The mock opens the "tunnel" ports when a task is submitted. 39 of 39 checks passed:
    • No token: up, selftest and submit of both launchers end after their first request with the 401 "needs a token" message.
    • Token outside the compute group: the 403 message.
    • Invalid token: the "invalid or has expired" message; status then exits with an error.
    • With a token and an http:// TES_URL that is not local, the scripts refuse to send it.
    • Valid token: selftest, up, status, logs, print, cancel, restart and down work for tes-service.sh and omop-tesk-fi.sh. The OMOP up waits until Atlas, WebAPI and the EUNOMIA source answer.
    • up with TES_TOKEN set replaces the keep-alive loop (new pid, old one gone).
    • After the token is revoked, both loops log the expiry message, and logs fails with it.
    • With the expired token, restart fails and leaves the running service and its loop alone; down stops the loop, restarts the hub (lock-out) and warns that it didn't cancel.
    • A sampler read /proc/*/cmdline every 20 ms during the whole run. The token appeared in no process's argv; a positive-control process with the token in its argv was detected.
    • No token material in .state files or logs.
    • The stub hub never ran with TES_TOKEN in its environment.
    • The harness is not in the repo, which has no fitting test suite for shell scripts. It is kept in the work folder: checks/docs-134-tes-tokens/run.sh and mock_tes.py.
  • The install block of the commands page, with its snippets expanded and run in a temporary HOME, recreates tes-api.sh, tes-service.sh, tes-hub.sh, common.sh, omop-tesk-fi.sh and pod/db.sh byte for byte.
  • python scripts/check_names.py: OK. mkdocs build --strict: OK. pytest: 25 passed, 100% coverage.
  • CI on the PR: CI passed (PR title, Docs, Test), CodeQL and the Read the Docs preview are green.
  • Not run: the scripts against a real TES service with tokens (issue checklist item 6). That needs the new instance address and a token. See the open questions.

Declined review findings

  • Move stop_watch (and the guard case) into tes-api.sh (low). The launchers already carry their own copies of small helpers (watching, is_done, lock_out), and tes-api.sh is about TES requests.
  • Move the mock-test note out of the page (medium, quality). It belongs in the page's existing "How this page was tested" section, which already says what was and wasn't tested, and it says plainly that no real TES was used.
  • Send no header from the commands-page curl checks when TES_TOKEN is unset (low). Those commands come after "Set the access token". The scripts themselves send no header without a token.
  • After SIGTERM, the loop leaves its background sleep until it times out (low): harmless.
  • A hint in the wait loop of up after a down without cancel (low): the docs say to run cancel before up instead.
  • read -rsp is bash-only (low). The tutorial runs on a VM with bash, and the step now says "in the shell (bash)".

Sourcery threads: 4 fixed in 1c948ef (HTTPS only with a token, curl -q, early un-export, status exit code). The other 7 were answered in their threads and resolved: xtrace, cancel || true, the restart window, overlapping up runs, down going on, buffered logs, and the "without tokens" wording.

Open questions

  1. New TES address. The address of the TES instance with token access isn't known yet. The default TES_URL stays https://csc-tesk-noauth.rahtiapp.fi. It is marked "to be updated" in both launchers, and a note on the page says it will change. Once the address is known, these need updating: the defaults in tes-service.sh and omop-tesk-fi.sh, the settings table, the commands page (settings block and both curl commands), and the note and the "without tokens" wording in "What you need". Then the scripts should run once against it with a token (checklist item 6). Options:
  2. How to join the compute group. The page tells readers to write to cloud-service@elixir-europe.org to join ECP_CLN:EOC-Compute. Is that the intended way?

🤖 Generated with Claude Code

https://claude.ai/code/session_011FSoqMpmDCfbACLj3uy6xV

Summary by Sourcery

Enable the TES tutorials to securely operate with token-authenticated services while documenting setup, renewal, and failure handling.

New Features:

  • Add optional TES access-token support to both tutorial launchers, including secure bearer-token handling and early authentication checks.
  • Document obtaining, configuring, renewing, and troubleshooting LS Login access tokens for TES services.

Bug Fixes:

  • Prevent access tokens from leaking into task environments, child processes, command lines, files, or logs.
  • Handle expired or rejected tokens safely in keep-alive loops and allow down to lock tasks out even when cancellation cannot reach the TES.

Enhancements:

  • Centralize authenticated TES requests and ensure keep-alive loops use refreshed tokens when services are brought up again.
  • Update tutorial prerequisites, settings, installation snippets, commands, and security guidance for token-protected TES deployments.

Documentation:

  • Add token usage instructions, compute-group requirements, HTTP status guidance, token lifetime information, and updated TES endpoint notes to the interactive services tutorials.

Tests:

  • Document verification against a token-enforcing mock TES service, including token privacy, loop behavior, and failure handling.

uniqueg and others added 4 commits November 12, 2026 16:53
tes-service.sh and omop-tesk-fi.sh send every request to the TES
through tes_curl (new tes-hub/tes-api.sh), which adds
"Authorization: Bearer $TES_TOKEN" when TES_TOKEN is set. The header
reaches curl through process substitution with the builtin printf,
so the token is never on a command line, in a file or in a log.

Commands that use the TES check access first and end with a clear
message on 401 (no, invalid or expired token) and 403 (not in the
compute group). The keep-alive loops log a rejected token and send
no new tasks; up with TES_TOKEN set replaces a running loop, so that
it gets the new token. The default TES_URL is marked to be updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a section on getting an LS Login access token, the compute group
ECP_CLN:EOC-Compute, the token lifetime of about 4 hours and the
meaning of 401 and 403, with a hidden prompt for TES_TOKEN. Send the
token in the curl commands of the commands page through stdin, add
tes-api.sh to the install block and the file table, add TES_TOKEN to
the settings, and replace the note that the TES has no tokens with a
generic one. Note that the default TES_URL will change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review findings: down and restart no longer fail early on a refused
token, because locking the old tasks out of the hub needs no TES;
down warns that it couldn't cancel. tes-api.sh un-exports TES_TOKEN,
and the launchers pass it to their keep-alive loop only, so that the
hub doesn't inherit it. The loop of tes-service.sh ends on SIGTERM
only between rounds, never inside lock_out. Docs: curl 7.55, bash
for read -s, the behaviour of down, clearer wording.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round 2 review: restart checks the TES before down, so that an
expired token leaves a running service alone. tes-hub.sh unsets
TES_TOKEN, also when started by hand. A keep-alive loop removes
the pid file only if it still holds its own pid, and stop_watch
fails instead of removing it while the old loop still runs. The
omop down takes the lock before it stops the loop. Docs: cancel
before up after a down without a valid token.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@uniqueg uniqueg self-assigned this Nov 12, 2026
@sourcery-ai

sourcery-ai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Implementation

Add the shared TES request helper that attaches tokens safely, enforces HTTPS when a token is present, and reports authorization failures.

  • docs/tutorials/tes-services/tes-hub/tes-api.sh L1-46

Route the single-container launcher’s TES operations through the helper, check access before TES-dependent commands, and manage token-aware keep-alive loops and shutdown.

  • docs/tutorials/tes-services/tes-hub/tes-service.sh L23-30, L166-173, L205-239 (+9 small edits)

Integrate token-authenticated TES requests into the OMOP launcher, including its access checks, keep-alive replacement, and down behavior when cancellation cannot reach the service.

  • docs/tutorials/tes-services/omop-tesk/omop-tesk-fi.sh L28-38, L239-257, L324-337 (+9 small edits)

Prevent the long-running hub process from inheriting the TES access token, including when the hub is started directly.

  • docs/tutorials/tes-services/tes-hub/tes-hub.sh L25-26

Documentation

Explain token setup, authorization errors, token expiry and loop behavior, and update prerequisites, security wording, settings and troubleshooting for the interactive tutorial.

  • docs/tutorials/tes-services/interactive_services_tes.md L86-96, L161-205, L395-406 (+6 small edits)

Update the command-by-command tutorial to install the shared helper, prompt for and use a token in TES curl examples, and refresh recovery and settings guidance.

  • docs/tutorials/tes-services/interactive_services_tes_commands.md L21-24, L92-104, L205-209 (+3 small edits)

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@read-the-docs-community

read-the-docs-community Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

@uniqueg
uniqueg marked this pull request as ready for review November 12, 2026 17:04
@sourcery-ai

sourcery-ai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Linked issues

Checked at commit e38b35f

#134 docs: send access tokens in the TES tutorials · 2 of 6 met · 2 to check by hand

# Requirement Status Notes
1 Send the bearer token on TES requests and fail clearly on 401 or 403. 🟡 Partly met TES requests use the shared token-aware client and most commands check access first, but down continues after a refused check so it can lock tasks out and warn that cancellation was skipped.
2 Keep the token out of command lines, shell history, files and logs. ✅ Met The tutorials use a hidden prompt, and the launchers keep the token unexported except when starting their keep-alive loop; TES requests pass the header through a pipe.
3 Explain how to get a token, how long it lives, and what 401 and 403 mean. ✅ Met The new token section covers LS Login, compute-group membership, the approximate lifetime, and both response codes.
4 Update the default TES_URL and variables table to the new address once it is known. 👀 Check by hand (external system) A person must confirm the token-enabled TES address has been published; the PR says it is not yet known and leaves the current default in place.
5 Remove notes that the default instance is unauthenticated. 🟡 Partly met The danger box is now generic, but the prerequisites still describe the default TESK as operating without tokens.
6 Run each script once against a token-enabled TES and note the result. 👀 Check by hand (runtime) A person must run the scripts against the real token-enabled TES and record the results; the reported checks used a mock, not a real TES.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 11 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="docs/tutorials/tes-services/tes-hub/tes-api.sh" line_range="18" />
<code_context>
+tes_curl() {
+  local out code
+  if [[ -n "${TES_TOKEN:-}" ]]; then
+    out="$(curl -sS -w '\n%{http_code}' -H @<(printf 'Authorization: Bearer %s\n' "$TES_TOKEN") "$@")" || return 1
+  else
+    out="$(curl -sS -w '\n%{http_code}' "$@")" || return 1
</code_context>
<issue_to_address>
**Bearer token sent to arbitrary TES URL**

When a user configures `TES_URL` as HTTP or points it at a host they do not trust, `tes_curl` sends `TES_TOKEN` to the URL in `TES_API` without checking its scheme or host, so a user who sets `TES_URL` to an HTTP endpoint or an untrusted HTTPS host exposes the bearer token to that endpoint. The commands-page `curl` examples do the same, allowing the recipient to use the token against the TES service.

Require HTTPS whenever a token is set, and document that `TES_URL` must identify the intended trusted TES service.

Also at `docs/tutorials/tes-services/interactive_services_tes.md:90`, `docs/tutorials/tes-services/interactive_services_tes_commands.md:102`, `docs/tutorials/tes-services/interactive_services_tes_commands.md:217`.
</issue_to_address>

### Comment 2
<location path="docs/tutorials/tes-services/tes-hub/tes-api.sh" line_range="18" />
<code_context>
+tes_curl() {
+  local out code
+  if [[ -n "${TES_TOKEN:-}" ]]; then
+    out="$(curl -sS -w '\n%{http_code}' -H @<(printf 'Authorization: Bearer %s\n' "$TES_TOKEN") "$@")" || return 1
+  else
+    out="$(curl -sS -w '\n%{http_code}' "$@")" || return 1
</code_context>
<issue_to_address>
**Verbose curl logs bearer tokens**

When the user's curl configuration enables verbose output, `curl` reads its user config by default, so a config that enables verbose output makes these requests print the outgoing `Authorization` header to stderr. The keep-alive loops redirect stderr to `watch.log`, persisting the token there.

Disable curl configuration for token-bearing requests (using `-q` as curl's first option), including the commands-page examples.

Also at `docs/tutorials/tes-services/tes-hub/tes-service.sh:192`, `docs/tutorials/tes-services/omop-tesk/omop-tesk-fi.sh:215`, `docs/tutorials/tes-services/interactive_services_tes_commands.md:102`, `docs/tutorials/tes-services/interactive_services_tes_commands.md:217`.
</issue_to_address>

### Comment 3
<location path="docs/tutorials/tes-services/tes-hub/tes-api.sh" line_range="11" />
<code_context>
+
+# The token stays a shell variable: not exported, so that no child process (such as the
+# hub, which runs for days) gets it. The launchers pass it to their keep-alive loop only.
+if [[ -n "${TES_TOKEN:-}" ]]; then export -n TES_TOKEN; fi
+
+# tes_curl <curl arguments>: prints the body of a 2xx answer. Otherwise prints why to
</code_context>
<issue_to_address>
**Shell tracing exposes the token**

When the Bash environment enables xtrace, such as through exported `SHELLOPTS` or `BASH_ENV`, bash xtrace prints expanded command arguments, including the token in the `TES_TOKEN` test and `printf` command; it also traces the token-bearing keep-alive launch assignment. The loop launch redirects stderr to `watch.log`, so the token is written to disk.

Disable xtrace before any token-bearing expansion or loop-launch assignment, and restore it only after those sensitive operations.

Also at `docs/tutorials/tes-services/tes-hub/tes-api.sh:18`, `docs/tutorials/tes-services/tes-hub/tes-service.sh:192`, `docs/tutorials/tes-services/omop-tesk/omop-tesk-fi.sh:215`.
</issue_to_address>

### Comment 4
<location path="docs/tutorials/tes-services/tes-hub/tes-service.sh" line_range="60" />
<code_context>

 die() { echo "ERROR: $*" >&2; exit 1; }
+# shellcheck source=SCRIPTDIR/tes-api.sh
+source "$HERE/tes-api.sh"
 json_get() { python3 -c 'import json,sys; d=json.load(sys.stdin); print(eval(sys.argv[1], {"d": d}))' "$1"; }
 ID_FILE="$STATE_DIR/task"
</code_context>
<issue_to_address>
**Child processes inherit the token**

When the user exports `TES_TOKEN` as instructed, the launchers run commands such as `mkdir`, `chmod`, and `sed` before sourcing `tes-api.sh`; those child processes inherit the bearer token before `export -n` runs, exposing it to programs that do not need it.

Un-export `TES_TOKEN` before launching any child process, while explicitly passing it only to the keep-alive loop.

Also at `docs/tutorials/tes-services/omop-tesk/omop-tesk-fi.sh:74`, `docs/tutorials/tes-services/tes-hub/tes-api.sh:11`.
</issue_to_address>

### Comment 5
<location path="docs/tutorials/tes-services/tes-hub/tes-service.sh" line_range="141" />
<code_context>
 cmd_cancel() {
   local id; id="$(task_id)"
-  [[ -n "$id" ]] && curl -fsS -m 30 -X POST "$TES_API/tasks/$id:cancel" >/dev/null && echo "canceled $id" || true
+  [[ -n "$id" ]] && tes_curl -m 30 -X POST "$TES_API/tasks/$id:cancel" >/dev/null && echo "canceled $id" || true
 }

</code_context>
<issue_to_address>
**Failed cancellations appear successful**

When the task-list check succeeds but a later cancel request fails, `cmd_cancel` and `cancel_role` suppress `tes_curl` failures with `|| true`, so `down` skips its uncanceled-task warning and exits successfully while tasks remain running; callers see cancellation as successful.

Propagate cancellation failures and have `down` emit its warning when cancellation fails.

Also at `docs/tutorials/tes-services/omop-tesk/omop-tesk-fi.sh:157-158`, `docs/tutorials/tes-services/omop-tesk/omop-tesk-fi.sh:254-255`, `docs/tutorials/tes-services/tes-hub/tes-service.sh:142`, `docs/tutorials/tes-services/tes-hub/tes-service.sh:219-220`.
</issue_to_address>

### Comment 6
<location path="docs/tutorials/tes-services/tes-hub/tes-service.sh" line_range="234" />
<code_context>
   down)     cmd_down ;;
-  restart)  cmd_down; cmd_up ;;
+  restart)  tes_check || die "cannot use the TES at $TES_URL"; cmd_down; cmd_up ;;
   setup)    cmd_setup ;;
   submit)   lock; for r in $(roles_arg "${2:-}"); do submit_role "$r"; done; unlock ;;
   status)   cmd_status ;;
</code_context>
<issue_to_address>
**Restart leaves the service inaccessible**

When TES access fails after the restart preflight, `cmd_down` locks out the old tasks before `cmd_up` checks TES access; `current_state` or `role_state` turns that error into `UNREACHABLE`, which `up` treats as still running, so it starts a loop without replacement tasks and leaves the service inaccessible.

Make restart abort without locking out existing tasks when TES access fails, and make `cmd_up` fail rather than treat TES errors as a running state.

Also at `docs/tutorials/tes-services/omop-tesk/omop-tesk-fi.sh:332`.
</issue_to_address>

### Comment 7
<location path="docs/tutorials/tes-services/tes-hub/tes-service.sh" line_range="190" />
<code_context>
   local state i; state="$(current_state)"
   if is_done "$state"; then cmd_submit; else echo "task $(task_id) is $state, not resubmitting"; fi
+  # The loop keeps the token it started with: replace it, so that it gets the current one.
+  if watching && [[ -n "${TES_TOKEN:-}" ]]; then stop_watch; fi
   if ! watching; then
-    setsid -f "$HERE/tes-service.sh" "$DIR" watch >> "$STATE_DIR/watch.log" 2>&1 < /dev/null
</code_context>
<issue_to_address>
**Orphan loops keep submitting tasks**

When two token-bearing `up` invocations overlap while replacing or starting a keep-alive loop, both invocations act on stale watcher state and can launch duplicate loops with different tokens, overwriting the shared PID file; `down` then stops only the recorded loop, leaving an orphan able to submit tasks.

Serialize watcher stop-and-start operations and ensure only one keep-alive loop can run.

Also at `docs/tutorials/tes-services/tes-hub/tes-service.sh:192`, `docs/tutorials/tes-services/omop-tesk/omop-tesk-fi.sh:213-215`.
</issue_to_address>

### Comment 8
<location path="docs/tutorials/tes-services/tes-hub/tes-service.sh" line_range="227" />
<code_context>
+# locks the pods out first, and needs the TES only to cancel.
+case "${1:-}" in
+  up|submit|logs|cancel|selftest) tes_check || die "cannot use the TES at $TES_URL" ;;
+  status) tes_check || true ;;
+esac
+
</code_context>
<issue_to_address>
**Status succeeds without TES access**

When the TES rejects the task-list check because the token is missing, invalid, or expired, `status` ignores the `tes_check` failure with `|| true` and continues querying state, reports `UNREACHABLE`, then exits successfully; shell callers see a successful check despite the incomplete status report.

Make `status` return failure when `tes_check` fails, including in the OMOP launcher.

Also at `docs/tutorials/tes-services/omop-tesk/omop-tesk-fi.sh:326`, `docs/tutorials/tes-services/tes-hub/tes-service.sh:199`.
</issue_to_address>

### Comment 9
<location path="docs/tutorials/tes-services/omop-tesk/omop-tesk-fi.sh" line_range="255" />
<code_context>
-  local r; for r in "${ROLES[@]}"; do cancel_role "$r"; done
+  local r
+  if tes_check; then for r in "${ROLES[@]}"; do cancel_role "$r"; done
+  else echo "WARNING: tasks not canceled; run: $0 cancel" >&2; fi
   unlock
 }
</code_context>
<issue_to_address>
**Down continues after rejection**

The issue asks commands to fail early on 401 or 403. Down checks the TES only after stopping the loop and locking out tasks, then warns and continues; the single-container launcher follows the same behavior.
</issue_to_address>

### Comment 10
<location path="docs/tutorials/tes-services/tes-hub/tes-api.sh" line_range="18-22" />
<code_context>
+tes_curl() {
+  local out code
+  if [[ -n "${TES_TOKEN:-}" ]]; then
+    out="$(curl -sS -w '\n%{http_code}' -H @<(printf 'Authorization: Bearer %s\n' "$TES_TOKEN") "$@")" || return 1
+  else
+    out="$(curl -sS -w '\n%{http_code}' "$@")" || return 1
+  fi
+  code="${out##*$'\n'}"; out="${out%$'\n'*}"
+  case "$code" in
+    2??) printf '%s\n' "$out" ;;
</code_context>
<issue_to_address>
**Large task logs exhaust memory**

When a TES FULL response contains very large task logs, `tes_curl` captures the entire response in `out` before printing it; `logs` and `selftest` then parse FULL task responses, so a sufficiently large task log makes the shell and parser hold large copies at once and can exhaust VM memory before the output is produced.

Avoid buffering successful response bodies in `tes_curl`; preserve streaming while separately checking the HTTP status.
</issue_to_address>

### Comment 11
<location path="docs/tutorials/tes-services/interactive_services_tes.md" line_range="89" />
<code_context>
+- A TES service whose tasks can make outbound HTTPS connections, and an
+  access token for it if it needs one (see [Get an access
+  token](#get-an-access-token)). The scripts use the TESK of ELIXIR-FI
+  without tokens (`https://csc-tesk-noauth.rahtiapp.fi`). To use another
+  service, set `TES_URL`.
+
</code_context>
<issue_to_address>
**Default still described as tokenless**

The issue asks to remove notes that the default instance is unauthenticated. The prerequisites still describe the default TESK as “without tokens,” which conveys that same point.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 10 findings to address first, and if a bearer token were exposed or sent to an unintended recipient, someone could use it as the account to access the TES and submit or cancel tasks until it expires or is revoked. Reverting removes this token flow, but cannot undo actions taken while the token was valid.

Blocking findings: docs/tutorials/tes-services/tes-hub/tes-api.sh:18, docs/tutorials/tes-services/tes-hub/tes-api.sh:18, docs/tutorials/tes-services/tes-hub/tes-api.sh:11, docs/tutorials/tes-services/tes-hub/tes-service.sh:60, docs/tutorials/tes-services/tes-hub/tes-service.sh:141, and 5 more


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

uniqueg and others added 2 commits November 12, 2026 18:20
Sourcery review: with TES_TOKEN set, tes_curl refuses a TES_URL that
isn't HTTPS (plain HTTP only to 127.0.0.1 or localhost) and calls
curl with -q, so that a ~/.curlrc can't turn on verbose output with
the header. The launchers un-export TES_TOKEN first thing, before
they run any program. status exits with an error when the TES
refuses the token. The commands page uses curl -q too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@uniqueg
uniqueg merged commit c0e5d36 into main Nov 13, 2026
13 checks passed
@uniqueg
uniqueg deleted the docs/134-tes-tokens branch November 13, 2026 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: send access tokens in the TES tutorials

1 participant